Graphical Interaction in a Multimodal Interface

The Multi-Modal Interface for Man Machine Interaction with Knowledge Based Systems (MMI2) project† is developing an advanced human computer interface to support co-operative dialogue between users and knowledge based systems. The interface supports natural language, command language, graphical display, direct manipulation and gesture. Several tools are incorporated to allow the graphical visualisation of the state of the knowledge base, its suggested designs for local area computer networks, and interactive charts and graphs. The mechanism for designing and presenting these visualisations which also supports the interaction of the graphical mode with other modes is described in this paper.